Add AWS VPC Lattice SDK-Compat Parity (73 Operations) - #331
Conversation
Implement the full aws-sdk-go-v2/service/vpclattice control-plane surface against the in-memory driver — all 73 SDK operations, no stubs. Covers service networks, services, listeners, rules (incl. BatchUpdateRule), target groups and targets, the three service-network association types, resource configurations, resource gateways, resource endpoint associations, access-log subscriptions, auth and resource policies, domain verifications, and tagging. VPC Lattice is the emulator's first REST-JSON (awsRestjson1) service: operations are routed by HTTP method + URL path rather than an X-Amz-Target header. The handler claims its top-level path prefixes and dispatches by segment + method; identifiers accept a bare ID or a full ARN. Union-typed fields (listener defaultAction, rule match/action, target-group config, resource-configuration definition) are stored as raw JSON and echoed back verbatim. Each resource group has a real-SDK round-trip lifecycle test, plus provider unit tests covering error paths, scoping, and clone-on-read isolation.
NitinKumar004
left a comment
There was a problem hiding this comment.
Review — AWS VPC Lattice (73 operations, REST-JSON)
Strong, genuinely-complete implementation of a new wire family (the repo's first REST-JSON / awsRestjson1 service). Verified in an isolated worktree: 73 operations, exactly 1:1 with the vendored SDK (73 api_op_* = 73 driver methods = 73 handler routes), no stubs (grep TODO|FIXME|panic|501 → none), and wire-faithful — timestamps/member keys/error envelope/X-Amzn-Errortype match the SDK, union fields echo byte-faithful, and ID-or-ARN path params survive %2F decoding. Gate is green: build/vet/gofmt/go test ./.../-race/golangci-lint 0. Architecture pillars all pass (memstore, idgen ARNs, injectable clock, canonical errors, no globals, dual-factory wiring), concurrency is sound (single mutex, cross-store counts atomic, copy-on-write verified), and it's correctly AWS-only.
Requesting changes on one High and a few real Medium correctness gaps — the 73-op parity and routing are excellent; these are the semantics to close before merge. Details inline.
Should fix
- [High] Create-time
Tagsare silently dropped on every resource — noCreate*path writesin.Tags, so the standard AWS create-with-tags →ListTagsForResourceflow returns{}across ~10 resource types. Untested. - Delete has no cascade / no active-child guard — deleting a service network with live associations succeeds (real AWS →
ResourceInUse); service/listener/target-group deletes orphan their children. - Stale / missing association counts — counts include associations to already-deleted services, and
UpdateServiceNetworknever recomputes them (always returns 0). UpdateResourceConfigurationclobbersAllowAssociationToSharedon a partial update (unconditional bool assignment).Matchesshadows the S3 catch-all — a path-style S3 op on a bucket namedservices/tags/targetgroups/… is hijacked by VPC Lattice (registered before S3).- 3 of 73 ops have no round-trip test coverage.
Also (Low): snake_case filenames (access_logs.go, service_networks.go, resource_gateways.go, resource_configs.go, domain_verifications.go, target_groups.go per docs/STRUCTURE.md §3) · in-place mutation of stored objects is safe under the current Mutex but would race if switched to RWMutex (document or clone-then-Set) · Register/Deregister-Targets discard the driver's failure list (always report all-success) · Delete{Auth,Resource}Policy return nil for a missing key · no name-uniqueness / clientToken idempotency on create · UpdateTargetGroup can't clear a health check · AccessLogSubscription.ResourceARN blanked from a bare ID · ListTargets unsorted · trailing path segments silently ignored · banner-style separator comments.
Excellent work on the REST-JSON routing and union fidelity — this is the correctness/AWS-semantics polish.
| CreatedAt: m.now(), | ||
| LastUpdatedAt: m.now(), | ||
| } | ||
| m.serviceNetworks.Set(id, sn) |
There was a problem hiding this comment.
[High] CreateServiceNetwork builds the record from in.Name/in.AuthType/… but never reads in.Tags — and no Create*/Start* method in the package writes to m.tags (repo-wide .Tags grep in the non-tagging sources returns nothing). So CreateServiceNetwork(&{Name:"x", Tags:{"a":"1"}}) → ListTagsForResource(sn.ARN) returns {} instead of {"a":"1"}, across ~10 resource types. This breaks the standard AWS create-with-tags → list-tags flow and is untested (the tag test only exercises standalone TagResource). Write in.Tags into m.tags on every create.
There was a problem hiding this comment.
[High] Fixed. Every Create*/Start*/Associate* now persists its tags via a shared writeTags(arn, tags) helper (create-with-tags → ListTagsForResource now round-trips across all resource types). The association/access-log/domain-verification creates that previously ignored the tag param (_ map[string]string) now thread it through. Added TestCreateWritesTags.
| return &out, nil | ||
| } | ||
|
|
||
| func (m *Mock) DeleteServiceNetwork(_ context.Context, identifier string) error { |
There was a problem hiding this comment.
DeleteServiceNetwork removes the SN even when VPC/service/resource associations still reference it — real AWS returns ResourceInUse. After delete, Get*Association still returns an association whose ServiceNetworkID points at a deleted network. Same class: DeleteService orphans its listeners/rules + SN↔service associations, DeleteListener orphans its rules, DeleteTargetGroup leaves listener/rule default-actions dangling. Either block on live dependents (ResourceInUse) or cascade them (as DeleteTargetGroup does for its targets).
There was a problem hiding this comment.
Fixed. DeleteServiceNetwork now returns ConflictException (FailedPrecondition) when any VPC/service/resource association still references it; DeleteService blocks on live SN↔service associations and cascades its listeners+rules; DeleteListener cascades its rules. (DeleteTargetGroup already cascades targets.) Added TestServiceNetworkDeleteBlockedByAssociation and TestServiceDeleteGuardAndCascade.
| return &out, nil | ||
| } | ||
|
|
||
| func (m *Mock) UpdateServiceNetwork( |
There was a problem hiding this comment.
UpdateServiceNetwork returns the record without calling applyAssocCounts, so its response always reports NumberOfAssociated{Services,VPCs,Resources} = 0 regardless of real associations (Get/List do compute them). Also, applyAssocCounts counts associations to services that may already be deleted (see the no-cascade delete above), so a GetServiceNetwork can report a phantom associated service. Call applyAssocCounts here, and skip associations whose target no longer exists.
There was a problem hiding this comment.
Fixed. UpdateServiceNetwork now calls applyAssocCounts, and applyAssocCounts skips SN↔service / SN↔resource associations whose target no longer exists (so a Get can't report a phantom). Covered by TestServiceNetworkAssocCounts (asserts the count drops to 0 after the target resource is deleted).
| c.Definition = append([]byte(nil), in.Definition...) | ||
| } | ||
|
|
||
| c.AllowAssociationToShared = in.AllowAssociationToShared |
There was a problem hiding this comment.
AllowAssociationToShared is assigned unconditionally, unlike the guarded PortRanges (!= nil) and Definition (len > 0) just above. It's a plain bool with no "unspecified" sentinel, so a partial UpdateResourceConfiguration that omits the flag silently resets a previously-true value to false. Guard it (e.g. accept *bool in the input, or only apply when the caller set it).
There was a problem hiding this comment.
Fixed. AllowAssociationToShared is now *bool in UpdateResourceConfigurationInput (threaded through the server decode). A partial update that omits it (nil) leaves the stored value unchanged; an explicit false clears it. Added TestUpdateResourceConfigKeepsAllowFlag.
| } | ||
|
|
||
| // Matches claims requests whose first path segment belongs to VPC Lattice. | ||
| func (h *Handler) Matches(r *http.Request) bool { |
There was a problem hiding this comment.
Matches claims any request whose first path segment is one of ~15 generic words (services, tags, targetgroups, servicenetworks, …), and this handler is registered before S3 (vpclattice at aws.go:287, S3's REST fallback at :438). Because aws-sdk-go-v2 uses path-style S3 against a custom endpoint, a path-style S3 op on a bucket literally named services/tags/etc. (GET /tags, PUT /services/mykey) is hijacked here and fails as a Lattice op — a real cross-service regression this PR introduces. Contrast Lambda, which avoids this by claiming a versioned prefix (/2015-03-31/functions). Consider gating on method+shape, or claiming a more distinctive path root.
There was a problem hiding this comment.
Fixed. Matches now gates on path-root + method + segment shape, not just the first segment. Verbs Lattice doesn't use for a root are declined (so S3 PUT /services/{key} falls through), and the ARN-remainder roots (tags/authpolicy/resourcepolicy) require the identifier segment (so S3 GET /tags on a bucket named tags falls through). Added TestMatchesDoesNotShadowS3. Documented the one residual ambiguity that's unavoidable for two REST services on one endpoint: identical verb+path like GET /services (list-services vs. S3 list-bucket-"services").
| } | ||
|
|
||
| // SN ↔ Service | ||
| svcA, err := client.CreateServiceNetworkServiceAssociation(ctx, &awsvpcl.CreateServiceNetworkServiceAssociationInput{ |
There was a problem hiding this comment.
The harness drives 70/73 ops but never asserts client.GetServiceNetworkServiceAssociation / GetServiceNetworkResourceAssociation (both are created here but only List/Delete are checked), and DeleteResourceEndpointAssociation is unreachable (no endpoint association is ever synthesized). The handlers exist, so the 73-op claim stands, but these Get paths have zero wire-level coverage. Cheap to close: add Get*Association asserts right after the creates.
There was a problem hiding this comment.
Fixed. Added wire-level asserts for GetServiceNetworkServiceAssociation and GetServiceNetworkResourceAssociation right after the creates, and a DeleteResourceEndpointAssociation call asserting ResourceNotFoundException (the endpoint-association surface isn't synthesized, so this exercises the otherwise-unreachable handler). All 73 ops now have wire-level coverage.
| @@ -0,0 +1,107 @@ | |||
| package vpclattice | |||
There was a problem hiding this comment.
[STRUCTURE.md §3] Smashed multi-word filenames should be snake_case: accesslogs.go → access_logs.go, and likewise service_networks.go, resource_gateways.go, resource_configs.go, domain_verifications.go, target_groups.go — in BOTH providers/aws/vpclattice/ and server/aws/vpclattice/ (a feature keeps the same filename across layers). New services now go through the STRUCTURE.md loop, so worth doing here.
There was a problem hiding this comment.
[STRUCTURE.md §3] Fixed. Renamed to snake_case in both layers: access_logs.go, service_networks.go, resource_gateways.go, resource_configs.go, domain_verifications.go, target_groups.go (via git mv).
Resolve the CHANGES_REQUESTED review on stackshy#331: - [High] every Create*/Start*/Associate* now persists in.Tags, so the standard create-with-tags → ListTagsForResource flow round-trips across all resources. - Deletes: DeleteServiceNetwork/DeleteService block on live associations (ConflictException); DeleteService cascades its listeners+rules; DeleteListener cascades its rules. - UpdateServiceNetwork now recomputes association counts; applyAssocCounts skips associations whose target service/resource no longer exists (no phantoms). - UpdateResourceConfiguration.AllowAssociationToShared is *bool — a partial update no longer resets a previously-true value. - Matches gates on method+shape so path-style S3 ops on buckets named like Lattice roots (PUT /services/key, GET /tags) fall through to the S3 catch-all. - snake_case filenames (access_logs, service_networks, resource_gateways, resource_configs, domain_verifications, target_groups) in both layers. - Register/Deregister wire the driver failure list; ListTargets sorted; AccessLogSubscription.ResourceARN no longer blanked from a bare ID. Adds provider unit tests + a Matches shadow test + wire coverage for the three previously-unasserted association Get/Delete ops. Full gate green.
…to feat/aws-vpclattice-parity # Conflicts: # docs/services.md # providers/aws/aws.go # server/aws/aws.go
|
Thanks — the High + all Medium items are fixed (replied inline). On the Also (Low) list:
Deferred, with reasoning (both genuinely Low / not standard AWS behavior):
Also synced the branch with Full gate green after all changes + the merge: |
NitinKumar004
left a comment
There was a problem hiding this comment.
Re-review — VPC Lattice fix commit
Great turnaround — verified against the code, H1, M2, M3, M4, M6 and the snake_case rename are all resolved, most exactly right:
- H1 — all 12
Create*/Start*nowwriteTags. - M2 —
DeleteServiceNetwork/DeleteServiceguard active associations (FailedPrecondition);DeleteService/DeleteListenercascade their children. - M3 —
UpdateServiceNetworknow callsapplyAssocCounts. - M4 —
AllowAssociationToSharedis now*bool+ nil-guarded (the right nil-sentinel fix). - M6 — both
Get*Associationare now driven via the real SDK client. - Naming — 6 files snake_cased in both layers; the CI Structure check passes.
Gate is green (build/vet/go test/-race/golangci-lint 0/Structure) with ~135 lines of new tests.
Still blocking — M5 (S3 shadowing) is only partially fixed
latticeClaims helps the ambiguous roots (bucket-level GET /tags and all PUTs now fall through to S3), but the core collision remains for the default roots: because isLatticeMethod includes GET/DELETE, an S3 object op like GET /services/mykey or DELETE /targetgroups/mykey on a bucket named exactly services/targetgroups/servicenetworks/… is still claimed by this handler and fails as a Lattice op. A user with such a bucket (registered before S3) still can't read/delete its objects. Details inline — requesting this be closed (or the residual explicitly bounded) before merge.
| return len(rest) >= 1 && | ||
| (method == http.MethodGet || method == http.MethodPost || method == http.MethodDelete) | ||
| default: | ||
| return isLatticeMethod(method) |
There was a problem hiding this comment.
The latticeClaims fix mitigates the ambiguous roots (tags/authpolicy/resourcepolicy now require an identifier segment, and PUT is excluded), but the default case still claims any GET/POST/PATCH/DELETE whose first segment is a Lattice root. So a path-style S3 object op — GET /services/mykey, DELETE /targetgroups/mykey, GET /servicenetworks/mykey — on a bucket literally named services/targetgroups/servicenetworks/etc. is hijacked here (VPC Lattice is registered before S3) and fails. The narrow-but-real residual: a bucket named exactly one of the ~15 roots can't do object GET/DELETE.
To close it robustly, gate the resource-id path on a VPC Lattice identifier shape — e.g. in routeByID, only claim when the id segment is a known Lattice ID prefix (sn-/svc-/tg-/listener-/…) or a arn:aws:vpc-lattice: ARN; otherwise fall through so S3 handles it (an S3 key like mykey won't match). That preserves all real Lattice paths while letting reserved-name buckets through. Alternatively, if a full fix isn't worth it, document the reserved bucket names as a known limitation so it's an explicit boundary rather than a silent mis-route.
There was a problem hiding this comment.
Fixed exactly as suggested — resource-scoped routes now gate on identifier shape. Matches claims a /<root>/{id} path only when {id} is Lattice-shaped: a generated ID prefix (sn-/svc-/tg-/listener-/rule-/snva-/snsa-/snra-/rcfg-/rgw-/als-/dv-/rea-) or contains a vpc-lattice ARN (isLatticeIdentifier). So GET /services/mykey, DELETE /targetgroups/mykey, GET /servicenetworks/mykey on a like-named S3 bucket now fall through to the S3 catch-all; real Lattice ops (ids/ARNs) are unaffected. The tags/authpolicy/resourcepolicy roots require the same Lattice id/ARN, so GET /tags/mykey also falls through.
The only residual is a bare GET /<root> (list) colliding with an S3 list-bucket on an identically-named bucket — unavoidable for two REST services on one endpoint — now called out explicitly in docs/services.md.
TestMatchesDoesNotShadowS3 extended with the object-op cases (GET/DELETE /<root>/mykey → not claimed; /<root>/<lattice-id> → claimed). Gate green: build/vet/gofmt/-race/golangci-lint 0. (541d4a6)
…pe (M5)
The re-review flagged that the earlier latticeClaims fix still let a path-style
S3 object op on a bucket named exactly like a Lattice root through — e.g.
`GET /services/mykey` or `DELETE /targetgroups/mykey` were claimed by this
handler (registered before S3) and failed, so such a bucket couldn't do object
GET/DELETE.
Resource-scoped routes now require the id segment to be Lattice-shaped (a
generated ID prefix like sn-/svc-/tg-/listener-/rule-/… or a vpc-lattice ARN);
an arbitrary S3 key ("mykey") no longer matches and falls through to the S3
catch-all. The tags/authpolicy/resourcepolicy roots likewise require a
Lattice id/ARN identifier. The one residual — a bare `GET /<root>` list vs. an
S3 list-bucket on an identically-named bucket — is unavoidable for two REST
services on one endpoint and is now documented.
Extended TestMatchesDoesNotShadowS3 with the object-op fall-through cases
(GET/DELETE /<root>/mykey → not claimed; /<root>/<lattice-id> → claimed).
|
Thanks for the re-review. M5 is now closed (the one remaining blocker): resource-scoped routes gate on a VPC Lattice identifier shape, so path-style S3 object ops ( All other items from the reviews (H1 tags-on-create, M2 delete cascade/guards, M3 counts, M4 |
NitinKumar004
left a comment
There was a problem hiding this comment.
Approving — all review findings resolved
Verified the M5 fix (541d4a69) in an isolated worktree at the PR head; gate green (build/vet/go test -race/TestMatchesDoesNotShadowS3 all pass).
M5 (S3 shadowing) is now genuinely closed. Resource-scoped routes are claimed only when the id segment is Lattice-shaped (a known ID prefix or a vpc-lattice ARN), so path-style S3 object ops on a like-named bucket (GET /services/mykey, DELETE /targetgroups/mykey) fall through to the S3 catch-all. I cross-checked isLatticeIdentifier's 13 prefixes against every ID the provider actually generates — exact match, no gaps — so no create→get-by-returned-id round-trip can wrongly fall through. The one prefix-less route (servicenetworkvpcendpointassociations) is list-only, so it correctly needs none. The single remaining residual — a bare GET /<root> list vs. an S3 list-bucket on an identically-named bucket — is inherent to two REST services on one endpoint and is now honestly documented in docs/services.md and the Matches comment.
That closes every finding from the earlier reviews — H1 (create-time tags), M2 (delete guard/cascade), M3 (assoc counts), M4 (AllowAssociationToShared nil-sentinel), M5, M6 (Get*Association coverage), and the snake_case rename. Clean, complete work — thanks for the thorough turnaround. LGTM.
Objective
Add full
aws-sdk-go-v2/service/vpclatticeSDK-compat parity to the emulator: all 73 operations, no stubs, so real VPC Lattice clients work end-to-end against the in-memory driver.What we found
The emulator had no VPC Lattice service, and — more significantly — no REST-JSON service at all. Every existing AWS service uses either AWS JSON 1.1 (
X-Amz-Targetheader) orawsquery. VPC Lattice speaks REST-JSON (awsRestjson1): operations are selected by HTTP method + URL path (POST /services,GET /services/{id}/listeners/{id},PATCH /servicenetworks/{id}, …), with URI path parameters that may be bare IDs or full ARNs.Blast radius: purely additive. New packages under
services/vpclattice/,providers/aws/vpclattice/,server/aws/vpclattice/, wired into the existing provider and server bundles. No existing service is touched.How we fixed it
Standard 4-layer pattern (driver → provider → server), following the Bedrock precedent for path-based REST routing:
map[string]segmentHandlerinNew();Matchesgates on path-root + method + segment shape (not just the first segment) so path-style S3 requests on like-named buckets fall through to the S3 catch-all, andServeHTTPdispatches by first segment + method. ReusablerouteCollection/routeByIDhelpers keep each resource group short. Identifiers accept ID-or-ARN; the three ARN-in-path surfaces (/authpolicy,/resourcepolicy,/tags) reconstruct the full ARN from the remaining path segments.defaultAction, a rule'smatch/action, a target group'sconfig, a resource configuration'sresourceConfigurationDefinition— are stored as raw JSON and echoed back verbatim, so any variant round-trips without modeling every shape.ListTagsForResourceround-trips); deletes block on live service-network associations (ConflictException) and cascade contained children (service→listeners→rules); association counts recompute on read and skip targets that no longer exist;UpdateResourceConfiguration.AllowAssociationToSharedis*boolso a partial update never resets it.UpdateTargetGroupmerges the health-check into the stored config.Alternatives not taken
net/http.ServeMuxpath patterns — its single-segment{id}matching mishandles ARNs-with-slashes (esp./tags/{resourceArn}); manual segment parsing (the Bedrock precedent) is robust to them.GET /services(Lattice list-services vs. S3 list-bucket-"services"). The method+shape gate removes every other collision.ACTIVE/PENDINGstatus, matching the other emulated services.Docs / Tests
docs/services.md— new "## 26. Application Networking" section (per-family op table + accepted-but-not-simulated notes), master-table row 26, summary count (+73). (Coexists with the Route 53 Resolver service that landed ondevelopment; combined Grand Total 1707.)aws-sdk-go-v2client throughhttptest— with wire-level coverage for all 73 ops and aMatchesS3-shadow test — plus a provider unit suite covering tags-on-create, delete guards/cascade, association-count recompute, error paths, ID/ARN resolution, scoping, batch partial-failure, and clone-on-read isolation.Test plan
go build ./...go vet ./...gofmtcleango test -race ./.../vpclattice/...— all pass (provider 80.3%, server 68.8%)golangci-lint run --new-from-rev=$(git merge-base HEAD stackshy/development) ./...— 0 issuesgo mod tidystable (addsvpclattice v1.25.5)Risk & Rollback
Low risk — additive only; no change to existing services or shared wire code. Rollback = revert this commit / drop the three new packages and their two wiring hunks.
Conclusion
VPC Lattice reaches full 73/73 SDK-compat parity and establishes the REST-JSON routing pattern for future REST services in the emulator.